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Pell Grants are the cornerstone of federal financial aid for low-income students enrolled in 
postsecondary education. Currently, these grants are available only to those who seek an initial 
undergraduate degree or credential lasting at least a typical semester. Because these rules may 
restrict access to programs providing skills needed for new or better jobs, the U.S. Department of 
Education (ED) began pilots of two experimental expansions to Pell Grant eligibility in 2011. The first 
experiment allowed income-eligible students with a bachelor’s degree to obtain Pell Grants for short- 
term occupational training programs. The second experiment allowed income-eligible students to 
obtain Pell Grants for very short-term programs lasting as little as eight weeks. This report presents 
the results from a rigorous evaluation of the experiments conducted by ED’s Institute of Education 
Sciences. The evaluation examined whether these pilot eligibility expansions increased enrollment in 
and completion of occupational training programs, a first step toward improving individuals’ success 
in the labor market. 


KEY FINDINGS 


e Offering Pell Grants for short occupational programs to low-income students with a bachelor’s 
degree increased program enrollment and completion by about 20 percentage points. 


e Offering Pell Grants for very short-term occupational training programs increased program 
enrollment and completion by about 10 percentage points. 


e More than half of students offered experimental Pell Grants used them, receiving an average 
grant amount of $1,800; they were just as likely as those not offered the grants to also use federal 
student loans. 


Each year, Federal Pell Grants help millions of low-income students pay for postsecondary 
education, including more than 7 million students in 2017-2018 alone (U.S. Department of Education 
2018). Eligibility rules are intended to ensure that funds support individuals most in need of financial 
aid and are targeted to educational programs substantial enough to provide a return on the federal 
investment. Thus, Pell Grants are available only to students with annual family incomes of up to 
about $50,000, who do not already have a bachelor’s degree, and who enroll in programs that last at 
least a typical semester (15 weeks).! However, the rules may prevent low-income adults who need a 
leg up in the labor market from benefiting from occupational training programs that can be 
completed in less time and often at a lower cost than other programs. 


Coming out of the Great Recession (2007-2009), policymakers and postsecondary schools sought 
ways for displaced workers to earn credentials that could quickly improve their job prospects. To 
address concerns that tuition and fees could be barriers to occupational training, ED decided to pilot 
test two expansions to Pell Grant eligibility. The pilots waived specific eligibility rules for a limited 
number of postsecondary schools that volunteered to participate. ED has the authority to waive 
federal financial aid regulations under the Experimental Sites Initiative of the Higher Education Act, 


in order to test policy ideas that might lead to changes in regulations or statutes. ED is required to 
evaluate each pilot or “experiment” and report the results to the U.S. Congress every other year.” 


The two 2011 Pell Grant experiments were intended to help low-income adults enter and complete 
short-term occupational training programs (Exhibit 1). Experiment 1 offered Pell Grants to post- 
bachelor’s students for short-term programs (for example, programs for registered nurses) and 
Experiment 2 offered Pell Grants for very short-term programs (for example, programs for certified 
nursing assistants). As is standard under current federal aid rules, the Pell Grants in both 
experiments had to be used for credit-earning programs leading to an educational certificate and the 
amounts were based on the program’s length and number of credits awarded.? 


Determining whether the experiments achieved their intended outcomes is important given the 
potential costs of making them permanent policy as well as questions about the economic benefits of 
short-term programs. This report presents the results from a rigorous evaluation of the experiments 
conducted by ED’s Institute of Education Sciences. The report describes the impacts of each 
experiment on eligible students’ enrollment, completion, and receipt of other forms of financial aid 
(see Appendix A for more information about the implementation of the 2011 Pell Grant experiments 
and Exhibit 2 for an overview of the evaluation design). 


Exhibit 1. The 2011 Pell Grant experiments 
Pilot experiments tested two expansions in Federal Pell Grant eligibility. 


Students were un- or underemployed and income-eligible for Pell Grants. 


Allowed programs included short-term, occupational programs leading to a certificate or 
credential aligned with local or regional workforce needs. 
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Exhibit 2. Overview of the evaluation design 

WHO PARTICIPATED? available (gender, in Experiment 2), which is about what 
would be expected by chance. This suggests the random 
assignment for each experiment worked as intended to 
create two statistically equivalent groups of students, such 
that any differences in their later outcomes can be 
interpreted as the result of being offered an experimental 
Pell Grant. 


Forty-six postsecondary schools volunteered, were approved by 
the U.S. Department of Education’s Federal Student Aid (FSA) 
Office to participate in the experiments, and identified eligible 
students.* Across both experiments, 72 percent were public two- 
year colleges and nearly half (46 percent) were in the southeastern 
region of the United States (Appendix A, Exhibit A.7). Thirty-five 
schools participated in Experiment 1, 28 schools participated in 


Experiment 2, and 17 participated in both experiments. 


In total, 2,684 adults were eligible for the experiments and 
were included in the analyses. To participate in the 
experiments, individuals had to meet all other Pell Grant 
eligibility criteria (such as being low income), fill out a 
FAFSA, and express interest in an occupational training 
program approved for the study. 


Experiment 1 (Pell for short-term programs for post-bachelor’s 
students): There are 414 students in these analyses. All had a 
bachelor’s degree,° 64 percent were female, and 93 percent 
were independent from their parents.° On average, they were 
age 36 and had a gross income of $20,670. Almost a quarter 
(24 percent) were already enrolled in a study school,’ though 
not in the program for which they hoped to receive an 
experimental Pell Grant, as required by the study (Appendix B, 
Exhibit B.4). 


Experiment 2 (Pell for very short-term-programs): There 
are 2,270 students in these analyses. About half (53 
percent) had some college education, 36 percent were 
female, and 85 percent were independent. On average, 
they were age 32 and had a gross income of $22,451. 
Fourteen percent were already enrolled in a study school 
(Appendix B, Exhibit B.4). 


HOW WAS THE STUDY CONDUCTED? 


In each school, eligible students were randomly assigned, 
separately by experiment, either to be offered or not 
offered experimental Pell Grant funds in their financial aid 
award packages. Students had a 60 percent chance of 
being offered experimental Pell Grant funds. Students in 
either group could receive any other financial aid for 
which they were eligible, as determined by study schools. 


The study compared outcomes for the two groups of 
students (those who were offered experimental Pell Grants 
and those who were not) in each experiment to measure 
the effects of the changes in Pell Grant eligibility. The two 
groups were similar on all but one of 18 characteristics 


WHAT OUTCOMES WERE MEASURED? 


The study focused on measuring enrollment and 
completion of any program in the study schools because 
students could only use experimental Pell Grant funds in 
the participating schools, yet could enter other programs 
that might also be beneficial to them. The primary 
outcomes were defined as follows: 


— Enrollment in any program at the study school within 
12 months of random assignment for Experiment 1 and 
8 months of random assignment for Experiment 2.° 


-— Completion of any program at study schools within 30 
months of random assignment for Experiment 1 and 10 
months of random assignment for Experiment 2." 


The study also examined exploratory outcomes (such as 
enrollment, completion, and federal student loan receipt 
at any school, including those outside of the study) to 
better understand the primary findings. Examining 
outcomes in any postsecondary institution allowed the 
study to assess whether students who did not have access 
to an experimental Pell Grant decided to enroll in another 
school instead. 


Within the time frame of the study, it was not possible to 
measure students’ employment and earnings due to the 
challenges of obtaining these data from other agencies that 
have labor market information for individuals." 


WHAT DATA WERE USED? 


The study drew on the following data sources: 


-— School records on student academic progress and 
attainment to measure primary enrollment and 
completion outcomes 


— National Student Clearinghouse data to explore 
enrollment and completion at schools other than those 
in the study?* 


— School records and administrative data from ED’s FSA 
databases to measure student characteristics and 
receipt of financial aid 


OFFERING PELL GRANTS TO STUDENTS WITH A BACHELOR’S DEGREE INCREASED 
PROGRAM ENROLLMENT AND COMPLETION 


With high unemployment rates among recent college graduates at that time," the post-bachelor’s Pell Grant 
experiment (Experiment 1) provided study participants the opportunity to use experimental Pell funds to 
pursue short-term occupational training lasting up to one year with full-time attendance. Lowering the cost 
barrier to these programs could allow low-income adults to invest in a new career or update their skills in 
their current field. Thus, the experiment sought to increase enrollment in and completion of certificate 
programs with value in the labor market. 


e Students with a bachelor’s degree were 26 percentage points more likely to enroll in additional 
education if they were offered an experimental Pell Grant to pay for a short occupational training 


program. Among Experiment 1’s college graduates, 78 percent of those offered experimental Pell Grants 
enrolled in some kind of program at a study school within a year, compared to 52 percent of students not 
offered experimental Pell Grants (Exhibit 3). The impact was also large (20 percent) when enrollment at 
any school was considered (Appendix C, Exhibit C.1), indicating that students not offered Pell Grants did 
not simply move to enroll in other, less expensive schools in large numbers. 


e Program completion increased by 17 percentage points. About half of the college graduates offered an 
experimental Pell Grant (52 percent) completed a program within 30 months at a study school compared 
to 36 percent of those not offered these funds (Exhibit 3). Students not offered experimental Pell Grants 
did not complete programs elsewhere in large numbers; nor does it appear that they completed longer 
programs that might have different, and perhaps better, employment or earnings prospects than the 
short programs the experiment encouraged (Appendix C, Exhibit C.1). 


Exhibit 3. Impact of offering experimental post-bachelor’s Pell Grants on enrollment in and 
completion of programs at study schools 
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SOURCE: School records. 

NOTE: Sample size is 414 students. The exhibit shows the percentage of students in the treatment group (“Offered experimental Pell Grant”) and 
in the control group (“Not offered experimental Pell Grant”) who were enrolled in any program at the study school within one year of random 
assignment, and who completed any program at the study school within 30 months of random assignment. The impact estimates are regression- 
adjusted for student socioeconomic characteristics measured before random assignment such that the percentage for the treatment group equals 
the sum of the unadjusted control group mean and the regression-adjusted impact estimate (percentages for completion of any program study 
schools do not add up due to rounding). See Appendix B for a description of the study’s analytic methods and Appendix C for full results. 

* Percentages differ significantly between the “Offered experimental Pell Grant” and “Not offered experimental Pell Grant” groups, 0.05 level, 
two-tailed test. 


Students with a bachelor’s degree who were offered an experimental Pell Grant were also 11 percentage 
points more likely to complete programs considered in high demand in their state. The offer of an 
experimental Pell Grant increased completion of not just any education program at study schools, but of 
high-demand programs—defined as those associated with occupations in a new and emerging field, 
projected to grow rapidly, or having a large number of openings in the student’s state. About 40 
percent of students offered an experimental Pell Grant completed a high-demand program, compared to 
29 percent of students who were not offered these funds (Exhibit 4). By far the most common high- 
demand programs completed by students offered an experimental Pell Grant were in the health 
professions (Exhibit 4), such as nursing and emergency medical technology. 


Exhibit 4. Completion of high-demand programs at study schools among students who were offered 
experimental post-bachelor's Pell Grants 


(a) Impact of offering post-bachelor's Pell Grants (b) Percentage of students who were 
on completion of high-demand programs offered an experimental Pell Grant and completed 
Nica a high-demand program, by field 
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SOURCE: School records, O*Net (https://www.onetonline.org/find/bright), National Center for Education Statistics 
(https://nces.ed.gov/pubs2002/cip2000/). 


NOTE: Panel a (sample size = 414 students) shows the percentage of students in the treatment group (“Offered experimental Pell Grant”) and in 
the control group (“Not offered experimental Pell Grant”) who completed any high-demand program at the study school within 30 months of 
random assignment. The impact estimates are regression-adjusted for student socioeconomic characteristics measured before random 
assignment such that the percentage for the treatment group equal the sum of the unadjusted control group mean and the regression-adjusted 
impact estimate. See Appendix B for a description of the study’s analytic methods and Appendix C for full results. Panel b (sample size = 144 
students) shows the percentage of students in the treatment group who completed each type of program, among those who completed a high- 
demand program. Program types with fewer than 10 students were grouped together into the “Other” category and include programs such as 
foreign languages, literatures, and linguistics; engineering technologies and engineering-related fields; personal and culinary services; and 
homeland security, law enforcement, firefighting, and related protective services. The study associated programs with specific occupations using 
crosswalks from the National Center for Education Statistics. To determine if a program was in a high-demand occupation in its state, the study 
adapted criteria established by the Occupational Information Network (O*NET) for “Bright Outlook” occupations that were expected to grow 
rapidly or have large numbers of job openings between 2014 and 2016, midway through the study period (see Appendix B for details on how the 
study identified high-demand programs). 


* Percentages differ significantly between the “Offered experimental Pell Grant” and “Not offered experimental Pell Grant” groups, 0.05 level, 
two-tailed test. 


e The experiment’s impacts on enrollment and completion were especially high among dislocated 
workers, but were consistent for students of different genders, ages, and incomes, and those facing 
different local unemployment conditions. The offer of an experimental Pell Grant for college graduates 
was particularly effective in increasing program enrollment and completion for the un- or 
underemployed students the experiment intended to help. Among students who identified themselves as 
a “dislocated worker” on the Free Application for Federal Student Aid (FAFSA), those who received an 
experimental Pell Grant offer were 46 percentage points more likely to enroll in a program than those 
who were not offered these funds. In contrast, the offer increased enrollment among nondislocated 
workers by about 14 percentage points. Dislocated workers also experienced larger impacts on program 
completion (28 versus 5 percentage points). Aside from the greater impacts among dislocated workers, 
the experiment was similarly effective for other groups of students, including those living in 
communities with different unemployment rates (Appendix C, Exhibit C.3). 


OFFERING PELL GRANTS FOR VERY SHORT-TERM OCCUPATIONAL TRAINING 
PROGRAMS ALSO INCREASED PROGRAM ENROLLMENT AND COMPLETION 


Short-term programs have become an increasingly popular path to obtaining a postsecondary credential, 
with the number of short, vocational credentials awarded more than doubling between 2000 and 2012 (Soliz 
2016). Students with family incomes low enough to qualify for Pell Grants may find certificates from very 
short programs (two to four months in length) especially appealing because the programs (1) tend to have 
lower cost, (2) more easily allow students to work while learning (U.S. Department of Education 2020), and 
(3) provide a credential that signals skills to employers more quickly than longer programs. In fact, some 
evidence indicates that short-term vocational credentials can lead to higher earnings than more traditional 
associate’s degrees, though the comparative benefits of these credentials vary. The FSA-approved programs 
eligible for Pell Grants under Experiment 2 were about 11 weeks long on average and primarily in the fields of 
transportation and materials moving, health professions, security and protective services, and mechanic and 
repair technologies (Appendix A, Exhibit A.11).1” As with Experiment 1, this experiment sought to increase 
enrollment in and completion of programs with value in the labor market. 


e Students who were offered an experimental Pell Grant to pay for a very short-term occupational 
training program were 15 percentage points more likely to enroll in additional education than students 
who did not receive the offer. In Experiment 2, 66 percent of students offered an experimental Pell Grant 
enrolled in any program at a study school within eight months, compared to 52 percent of students not 
offered an experimental Pell Grant (Exhibit 5). The size of the impact was similar (14 percentage points) 
when examining enrollment in any postsecondary school (Appendix C, Exhibit C.2). 


e Program completion increased by 9 percentage points. Nearly half of the students offered an 
experimental Pell Grant for a very short-term occupational training program (47 percent) completed a 
program within 10 months at a study school compared to 38 percent of those not offered these funds 
(Exhibit 6). There was no evidence that the significant effect on program completion would be different if 
the analysis were extended beyond study schools or beyond the 10-month follow-up window (Appendix 
C, Exhibit C.2). 


Exhibit 5. Impact of offering experimental Pell Grants for very short-term occupational training 
programs on enrollment in and completion of programs at study schools 
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SOURCE: School records. 


NOTE: Sample size is 2,270 students. The exhibit shows the percentage of students in the treatment group (“Offered experimental Pell Grant”) 
and in the control group (“Not offered experimental Pell Grant”) who were enrolled in any program at the study school within 8 months of 
random assignment, and who completed any program at the study school within 10 months of random assignment. The impact estimates are 
regression-adjusted for student socioeconomic characteristics measured before random assignment such that the percentage for the treatment 
group equal the sum of the unadjusted control group mean and the regression-adjusted impact estimate (percentages for enrollment in any 
program at study schools do not add up due to rounding). See Appendix B for a description of the study’s analytic methods and Appendix C for 
full results. 


* Percentages differ significantly between the “Offered experimental Pell Grant” and “Not offered experimental Pell Grant” groups, 0.05 level, 
two-tailed test. 


Students offered an experimental Pell Grant to pay for a very short-term occupational training 
program were 8 percentage points more likely to complete programs considered in high demand in their 
state. The offer of an experimental Pell Grant increased completion of programs associated with high- 
demand occupations in a student’s state. About 41 percent of students offered an experimental Pell Grant 
completed a high-demand program, compared to 33 percent of students who were not offered these 
funds (Exhibit 6). Among those students who were offered an experimental Pell Grant and completed a 
high-demand program, 65 percent completed programs in transportation and materials moving (Exhibit 
6). Almost all of these programs provided training in truck and commercial vehicle operation. 


The experiment’s positive impacts on enrollment and completion were consistent for students with 
different characteristics. The offer of an experimental Pell Grant was similarly effective in increasing 
enrollment and completion among students of different genders, ages, and incomes, as well as those 
facing different employment challenges and local unemployment rates (Appendix C, Exhibit C.4). 


Exhibit 6. Completion of high-demand programs at study schools among students who were offered 
experimental Pell Grants for very short-term occupational training programs 
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SOURCE: School records, O*Net (https://www.onetonline.org/find/bright), National Center for Education Statistics 
(https://nces.ed.gov/pubs2002/cip2000/). 


NOTE: Panel a (sample size = 2,270 students) shows the percentage of students in the treatment group (“Offered experimental Pell Grant”) and in 
the control group (“Not offered experimental Pell Grant”) who completed any high-demand program at the study school within 30 months of 
random assignment. The impact estimates are regression-adjusted for student socioeconomic characteristics measured before random 
assignment such that the percentage for the treatment group equal the sum of the unadjusted control group mean and the regression-adjusted 
impact estimate. See Appendix B for a description of the study’s analytic methods and Appendix C for full results. Panel b (sample size = 863 
students) shows the percentage of students in the treatment group who completed each type of program, among those who completed a high- 
demand program. Program types with fewer than 10 students were grouped together into the “Other” category and include programs such as 
precision production; homeland security, law enforcement, firefighting, and related protective services; mechanic and repair 
technologies/technicians; and family and consumer sciences/human sciences. The study associated programs with specific occupations using 
crosswalks from the National Center for Education Statistics. To determine if a program was in a high-demand occupation in its state, the study 
adapted criteria established by the Occupational Information Network (O*NET) for “Bright Outlook” occupations that were expected to grow 
rapidly or have large numbers of job openings between 2014 and 2016, midway through the study period (see Appendix B for details on how the 
study identified high-demand programs). 


* Percentages differ significantly between the “Offered experimental Pell Grant” and “Not offered experimental Pell Grant” groups, 0.05 level, 
two-tailed test. 


MOST STUDENTS OFFERED EXPERIMENTAL PELL GRANTS USED THEM BUT WERE JUST 
AS LIKELY AS THOSE NOT OFFERED PELL GRANTS TO ALSO USE FEDERAL LOANS 


When the two experiments began, it was expected that those most in need of training to get jobs were also 
those least able to pay for it out-of-pocket or to take out loans to cover the costs. Although economic 
conditions improved over the years that eligible students entered the experiments (2012-2017), the financial 
challenges many Pell Grant-eligible students face can still be substantial (Goldrick-Rab et al. 2018). It is 
therefore important to understand whether low-income students offered experimental Pell Grants used 
those funds and whether having access to these funds affected their need for additional aid to support their 
education. For example, having access to a Pell Grant could reduce a student’s need for a federal student 
loan to pay for a program. At the same time, because having access to Pell Grants encouraged more students 
to enroll in programs, more people may have needed additional funds to cover schooling costs. 


e More than half of students offered experimental Pell Grants used the funds, with about $1,800 
estimated to have been disbursed per student on average.* Surprisingly, many low-income students who 
took the time to apply to study schools, complete a FAFSA, express interest in experimental Pell Grants, 
and were offered an experimental Pell Grant ultimately did not take up this offer. Some did not enter a 
postsecondary program (see Exhibits 3 and 5). Some enrolled in programs without using the 
experimental Pell Grants—about 9 percent of those offered grants for Experiment 1 and 16 percent of 
those offered grants for Experiment 2. These programs students entered without experimental grants 
were either longer, not occupationally focused, or not credit earning—none of which were allowable 
under the experiments. Only two-thirds of eligible college graduates (67 percent) interested in additional 
occupational training and half (52 percent) of eligible students interested in very short-term programs 
went on to use the experimental Pell Grants offered to them (Exhibit 7). On average, those who did use 
the grants received $3,577 for short training programs (Experiment 1) and $1,312 for very short-term 
occupational training programs (Experiment 2), according to data from the study schools (Exhibit 7). 
Across the two experiments, the average amount students received was $1,752.!° 


Exhibit 7. Use of experimental Pell Grants among students who received the offer, by experiment 
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SOURCE: School records. 


NOTE: Panel a (sample size = 254 in Experiment 1 and 1,363 students in Experiment 2) shows the percentage of students offered experimental 
Pell Grants who went on to use the grants during the study period. Panel b (sample size = 170 students for Experiment 1 and 705 for Experiment 
2) shows the average experimental Pell Grant amount, per student, for those who used the grants. These amounts were reported by study 
schools and could cover more than one program and award year. 


Being offered an experimental Pell Grant had no impact on the share of students taking out federal 
student loans or the average amount of federal student loans they received. Federal loans can be used to 
pay for tuition, fees, books, and living expenses that are not already covered by Pell Grants or other aid. 
Despite the offer of a Pell Grant inducing significantly more students to enroll in postsecondary programs 
in both experiments, gaining access to experimental Pell Grant funding did not result in increased (or 
decreased) use of federal student loans. students who were offered experimental Pell Grants were just as 
likely as students without access to experimental Pell Grants to take out federal student loans and had 
similar loan amounts, on average (see Exhibits C.1 and C.2 in Appendix C).”° 


There are two factors that appear to have balanced out, resulting in no overall effect on students’ use of 
federal loans. If those offered and not offered experimental Pell Grants enrolled in the same types of 
programs in equal proportions, it is reasonable to expect that a higher share of those without access to 
the grants would rely on loans to cover costs. In fact, among the subset of students who enrolled in a 
program, a larger proportion of students not offered experimental funds took out loans (about 71 
percent) compared to students offered a Pell Grant (about 43 percent). But enrollment was not equal 
across the groups; students offered an experimental Pell Grant were 26 percentage points more likely to 
enroll in postsecondary programs in Experiment 1 and 15 percentage points more likely in Experiment 2 
(Exhibits 3 and 5). Some share of these additional enrollees took out federal loans, offsetting the higher 
proportion of students without access to these grants who used loans. 


The use of federal student loans was more common in Experiment 1, where programs were longer and thus 
costlier. Thirty-three percent of students with a bachelor’s degree who were offered experimental Pell Grants 
(Experiment 1) took out federal student loans; the average loan amount among these students was $12,950 
(Exhibit 8). Among students in Experiment 2 who were offered experimental Pell Grants for very short-term 
programs, only 9 percent took out federal loans, and the average disbursed amount was lower ($4,021; 
Exhibit 8). The fact that students with access to experimental Pell Grants took out loans is not surprising. The 
maximum Pell Grant award amount in 2014-2015 (about mid-way through the study period) was $5,730 (U.S. 
Department of Education 2015), far less than the average cost of attending a study school ($19,600 for 
Experiment 1 and $16,900 for Experiment 2).”) ? Even if the cost of attendance was prorated for the duration 
and credits earned in short- or very short-term programs, expenses likely remained for some. 


Exhibit 8. Use of federal student loans among students who received an offer of experimental Pell 
funds, by experiment 


(a) Percentage of students offered an experimental (b) Average federal loan 
Pell Grant who took out federal student loans disbursement amount (per student) 
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SOURCE: School records. 

NOTE: Panel a (sample size = 254 in Experiment 1 and 1,363 students in Experiment 2) shows the percentage of students offered experimental Pell 
Grants who went on to take out a federal student loan during the study period. Panel b (sample size = 87 students for Experiment 1 and 126 for 
Experiment 2) shows the average loan amount, per student, for those who used loans. These amounts were reported by FSA and could cover 
more than one program and award year. 
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LOOKING AHEAD 


ED’s spending on Pell Grants is now at close to $30 billion annually. Some policymakers and stakeholders seek to 
expand the amounts awarded or who is eligible. Others question the increasing costs of federal financial aid and 
whether the grants, and the investment in short-term programs in particular, encourage student success as 
intended (Kreighbaum 2019). It is therefore important to identify lessons for the future and still-open questions 
from this study, particularly given the potential costs of making the pilot eligibility expansions permanent policy. 
For example, if current very short noncredit occupational courses were eventually made credit-earning and eligible 
for Pell Grants, low-income students’ participation might add as much as $500 million more to federal financial aid 
expenditures. This cost should be balanced against a potential 10-percentage point impact on program 
completion.” 


Two important lessons stem from the way the study was conducted. This study was the first to evaluate the effects 
of offering Pell Grants using random assignment, the most rigorous methodology for assessing effectiveness. It was 
also the first time this method was used to evaluate a pilot under ED’s Experimental Sites Initiative, in place since 
the 1980s. The use of random assignment lends credibility to a growing body of evidence about the benefits of Pell 
Grants for low-income students. Other evaluations use different methods but some have also found that this form of 
aid leads to higher rates of college enrollment and completion, particularly among older, nontraditional students.*4 
The current study also underscores the feasibility of using random assignment in evaluating federal education 
programs, including financial aid, and in doing so on college campuses, as an increasing number of studies do. 


There are also two uncertainties about what might happen if the pilots did become permanent policy. 


e Would the findings be similar if the expansions in Pell Grant eligibility were extended to all postsecondary 
schools nationally, or at least all of those that offer short-term occupational programs? The participating 
schools were primarily public two-year colleges, concentrated in the southeastern United States. They tended 
to be either smaller or larger than similar schools in their state and had a larger percentage of Pell Grant 
recipients (see Appendix A). Their students had already filled out a FAFSA and expressed interest in a program 
of study at the school before learning about their eligibility for an experimental Pell Grant. These procedures 
limited the potential for information about the availability of the grants to spur more interest in short 
occupational programs and in applying for financial aid as might occur if the policy were expanded 
nationwide. Whether different schools, offering different programs, to different students would achieve the 
same success is unclear. 


e Do the effects on program enrollment and completion translate into the expected employment benefits? One 
important concern the study explored was whether the offer of Pell Grants for short occupational programs 
shifted students away from longer programs, such as an associate’s degree, that typically provide higher 
lifetime earnings Jepsen, Troske, & Coomes 2014). The exploratory analysis did not provide convincing 
evidence of reduced associate’s degree completion (Appendix C),”° yet students offered the grants were clearly 
and significantly more likely to complete certificate programs. Some studies suggest that certificates from 
certain short occupational training programs can be very attractive to employers, perhaps even more so than 
an associate’s or bachelor’s degree in a different field. However, the benefits of shorter programs seem to vary 
with the field of study, local labor market conditions, and individual characteristics like gender, race, and 
ethnicity (Carnevale, Rose, & Hanson 2012). 


Employment and wage data could shed light on the economic value of the occupational training programs students 
completed in these experiments. The data needed to conduct this analysis are available but could not be accessed 
during the study period. Thus, the labor market returns from the two experiments and how these compare to the 
cost of expanding Pell Grant eligibility remain important open questions for the future. 
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ENDNOTES 


'To be eligible for a Pell Grant, students must also meet the general federal student aid eligibility requirements. See U.S. Department of 
Education, Office of Federal Student Aid. (n.d.). Eligibility requirements. https://studentaid.gov/understand-aid/eligibility/requirements. 


? The Experimental Sites Initiative (ESI) is authorized by Section 487A(b) of the Higher Education Act of 1965 (20 U.S.C. 1094a(b)). 
Under the ESI statute, the Secretary of the U.S. Department of Education is required to review and evaluate the experiences of 
institutions that participate as experimental sites and, biennially, submit a report based on the review and evaluation to the authorizing 
committees (Section 487A(b)(2)). See U.S. Department of Education. 1998 Amendments to the Higher Education Act of 1965 


P.L. 105-244, Title IV-Student Assistance, Part G-General Provisions, Sec. 490. Regulatory Relief and Improvement. 
https://www2.ed.gov/policy/highered/leg/hea98/sec490.html. 


3 Experimental Pell Grants were prorated based on enrollment status and the length of attendance and were paid directly to institutions 
for participating students. Under the current rules, students cannot use Pell Grants for noncredit programs, which are generally shorter 
in duration than credit-earning programs but do not typically lead to an educational credential. See Reed, M. (2014, January 26). Credit 
for non-credit. Inside Higher Ed. https://www.insidehighered.com/blogs/confessions-community-college-dean/credit-non-credit. 


* Schools interested in participating in Experiments 1 and 2 went through a multi-step approval process. Schools interested in 
Experiment 2 required additional program approval before they could participate. Because very short-term programs are not typically 
authorized for federal financial aid, the programs had to be approved by the schools’ accreditor, the relevant state agency in some 
cases, and ED’s Federal Student Aid (FSA) Office. For more information on the approval process, see Appendix A. 


5 Although all those with a bachelor’s degree should appear in school or FSA records as having some prior postsecondary experience, 
records for the Experiment 1 sample indicated that approximately 6 percent did not have prior postsecondary experience. Despite this 
lack of documentation, schools determined these participants to be eligible for Experiment 1. 


6 To be considered independent on the FAFSA, applicants must meet at least one of the following criteria: be 24 years or older, have a 
dependent of their own, be married, be on active duty in the military or a veteran, be in foster care, an orphan, and/or a ward of the 
state, be homeless, or be an emancipated minor. https://studentaid.gov/apply-for-aid/fafsa/filling-out/dependency 


’ Prior enrollment was measured up to 12 months before random assignment. In both experiments, the differences in the percentage of 
students enrolled before random assignment did not differ between the group of students offered experimental Pell Grants and the 
group not offered experimental Pell Grants (Appendix B, Exhibit B.4). 


8 To examine the impact of each individual potential expansion to Pell Grant eligibility, the study compared students in the offered 
experimental Pell Grant group to those in the not-offered group separately for Experiment 1 and Experiment 2, using statistical 
(regression) models. These models took into account the demographic and academic characteristics of participating students and 
schools. See Appendix B for a more detailed description of the analyses that were conducted. 


° The time frame in which participants could enroll in a program was selected to be 12 months for Experiment 1 because the experiment 
allowed students up to two years to use the experimental Pell Grants toward a program up to one year in duration if completed full 
time. An 8-month follow-up period for measuring enrollment in Experiment 2 was selected because the minimum duration for a 
program in that experiment is eight weeks. 


’° Participants in Experiment 1 had up to two years to complete their program. Thus, the 30-month time period for follow-up was 
selected to allow for this two-year period plus an additional six months in the event participants did not enroll in a course immediately 
or took longer than expected to complete the program. Short occupational training programs are not generally offered on the same 
semester schedule as more traditional college courses; therefore, schools might have identified and entered eligible students into the 
lottery for experimental Pell Grants months before students enrolled in a program. A longer follow-up period would allow adequate 
time for students to enroll in and feasibly complete the very short programs eligible for Experiment 2. The study relies on school data, 
which includes student records from November 2012 through December 2017. Given that schools continued to identify students eligible 
for the study through March 2017, 10 months is the longest follow-up possible without substantially decreasing the sample size and 
statistical power of the study. 


"Tn addition, due to the relatively small number of participants, the study did not expect to have sufficient statistical power to detect 
impacts on employment and earnings. 
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” Research suggests that most state and national postsecondary data sets do not include data on noncredit students (Xu & Ran 2015). 
See Xu, D., & Ran, X. (2015). Noncredit education in community college: Students, course enrollments, and academic outcomes. 
https://ccere.tc.columbia.edu/media/k2/attachments/noncredit-education-in-community-college.pdf. 


3 Although overall coverage rates within the National Student Clearinghouse (NSC) have increased to more than 90 percent among 
public, not-for-profit schools, of the 46 study schools, 33 schools (72 percent) participated in the NSC. See Dundar, A., & Shapiro, D. 
(2016). The National Student Clearinghouse as an integral part of the national postsecondary data infrastructure. 
https://nscresearchcenter.org/wp-content/uploads/NSC-as-an-Integral-Part-of-the-National-Postsecondary-Data-Infrastructure.pdf. 


Tn October 2011, the unemployment rate among those who recently had completed bachelor’s degrees was 13.5 percent. See U.S. 
Bureau of Labor Statistics. (2013, April 5). The job market for recent college graduates in the United States. 
https://www.bls.gov/opub/ted/2013/ted_20130405.htm. 


5 Programs were classified as high-demand if they were associated with an occupation that met one or more of the following criteria 
established by the Occupational Information Network (O*NET) for the 2014-2016 period: was projected to have rapid growth or a large 
number of openings in the student’s state based on short-term occupational projections data, or was considered to be a new and 
emerging occupation. For more details, see Appendix A. 


'6 Students may qualify as a dislocated worker on the FAFSA if they lost their job, were laid off, or are receiving unemployment 
benefits. For a complete list of criteria, see https://fafsa.gov/fotw2021/help/fahelp69.htm. 


'’ The programs in which students expressed interest were not necessarily the same programs they enrolled in or completed. 
8 The maximum Pell Grant award ranged from $5,550 for the 2011-2012 school year to $5,920 for the 2017-2018 school year. 
'S These amounts were reported by study schools and could cover more than one program and award year. 


20 Data on loans students received during the study period, obtained from FSA’s administrative data systems, could span multiple 
award years, programs, and schools (including schools outside of the study) and therefore may not exactly line up with the period 
during which students used their experimental Pell Grants. Any misalignment would apply to those offered and not offered 
experimental Pell and therefore would not bias the estimates. 


2! The study did not obtain data on full- or part-time status from study schools. Students who enrolled less than full-time would have a 
lower cost of attendance but would also have their Pell Grant award amounts prorated based on the length of the program in which 
they enrolled. 


2 On average, the annual cost of attending a study school was about $18,000, but could range from approximately $9,000 to $46,000. 
Cost of attendance information was obtained from the 2014-15 Integrated Postsecondary Education System (IPEDS) and measures the 
total price for in-state students living off campus (not with family), as reported by schools. Twenty-two percent of schools in 
Experiment 1 and 25 percent of schools in Experiment 2 did not report this information in the 2014-15 IPEDS. 


23 See Appendix C, Exhibit C.15 for information on how we calculated this value. It is important to note that this is a rough estimate 
based on data from a variety of sources and may not reflect the true cost if these experiments were made official policy. 


4 For example, see Seftor and Turner (2002), Park & Scott-Clayton (2018), and Denning et al. (2019). Some quasi-experimental studies 
have also found limited impacts of Pell Grant eligibility, such as Carruthers and Welch (2019). Overall, the enrollment and completion 
impacts in the 2011 Pell Grant experiments are larger than in previous research on Pell Grants, which has typically focused on different 
types of students and programs. 


25Tn addition, students not offered experimental Pell Grants were no more likely to still be enrolled at the end of the study’s follow-up 
period, suggesting that they would not have completed a longer program if the follow-up period were extended (Appendix C). 
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